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Abstract 

Assessing student learning outcomes and determining achievement of the 
Interprofessional Collaborative Practice (IPCEP) Core Competency of 
Values/Ethics in a generic pre-professional Bachelor of Science in Health Science 
(BSHS) program is challenging. A course level Student Learning Outcome (SLO) 
is: “....articulate the impact of personal values and professional ethics in 
healthcare decision-making". A program level terminal learning outcome is to 
“....critically discuss the interface of values/ethics on health outcomes”. One 
university level Essential Learning Outcome (ELOs) that all baccalaureate 
students are expected to achieve by graduation is Ethical Reasoning. This was 
equivalent to the IPCEP Values/Ethics core competency. This paper describes a 
strategy to simultaneously measure the Values/Ethics competency at course, 
program, and university levels. A narrative analysis (n=94) using required ethical 
decision making BSHS student papers was conducted to determine achievement of 
SLOs/ELOs and the IPCEP Values/Ethics core competency. Eleven items in the 
grading criteria were linked to outcome criteria for university ELO competency. A 
point value was assigned to each of these items using a scoring rubric indicating 
level of achievement. Results indicated that most students were at the Skilled level 
for the majority of students, and demonstrated adequate achievement of university, 
program, and course level learning outcomes as well as achievement of the IPCEP 
Values/Ethics core competency. 
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The purpose of this project was to link university level Essential Learning Outcomes (ELOs) 
related to Ethical Reasoning with Program Learning Outcomes(PLOs) and Student Learning 
Outcomes (SLOs) and the Interprofessional Collaborative Practice Core Competency of Values 
and Ethics in an introductory baccalaureate level health science course. Assessment of a course 
level assignment using a rubric was used to determine mastery. The course is part of the Bachelor 
of Health Science (BSHS) Program in the School of Health sciences in a mid-Atlantic public 
liberal arts college with an enrollment of 8570 undergraduate and graduate students. 

Background 

Outcome based models of education are centuries old. One example is the apprentice-ship method 
common in the Middle Ages. Outcome-Based Education “means clearly focusing and organizing 
every-thing in an educational system around what is essential for all students to be able to do 
successfully at the end of their learning experiences. This means starting with a clear picture of 
what is important for students to be able to do, then organizing curriculum, instruction, and 
assessment to make sure this learning ultimately happen” (Spady, 1994, p. 12). 

Several conceptual models of instruction focus on ELOs and student competence (Cydis, et ah, 
2015). Malan (2000) described characteristics of the outcome based education. It is needs based, 
and outcome driven. It has a design down approach. It specifies outcomes and levels of outcomes. 
Malan (200) also stated that criterion based assessment of mastery or competencies is integral to 
outcomes based education. The focus is shifted from teaching to learning. The frame work is 
holistic in its outcomes focus, meaning that achievement of outcomes is not enough: they must be 
built upon continually. 

While the idea of outcome based learning is not new, the current trend toward instituting 
college and university-wide ELOs that provide an overriding structure aimed at providing a 
comprehensive, cohesive education across disciplines was influenced by the American 
Association of Colleges and Universities (AAC&U). The AAC&U launched the Liberal Education 
& America’s Promise (LEAP) program in 2005 to respond to the demands of the twenty first 
century for more college educated, engaged, and informed citizens through liberal education 
(AACU, n.d.). The LEAP initiative is organized around a group of ELOs that are thought to be 
best developed through a liberal education (AAC&U, 2007). LEAP developed Valid Assessment 
of Learning in Undergraduate Education (VALUE) rubrics with which to assess these ELOs on a 
campus wide basis. 

Our university developed their own ELOs based on AAC&U recommendations and drawn 
from teaching experience of our faculty. ELOs to be achieved during student experience at the 
university were designed to help all students “focus on the intellectual and marketable talents 
needed to prepare for personal and professional success in the 21st century” (Stockton University, 
n.d.). Several conceptual models of instruction focus on ELOs and student competence (Cydis, et 
al., 2015). Cydis, et al. (2015) described the integration and assessment of ELOs. They conducted 
mixed methods research and developed a model of ELO integration based on data collected at our 
university. They identified several components of ELO integration that converged to support 
student competence. These included the following concepts: Connection - involves students 
making associations among ELOs and understanding the process of change over time; Awareness 
- emphasizes the importance of metacognition regarding learning how to leam which in turn, 
supports lifelong learning; Facilitation-involves faculty facilitating ELO acquisition by clearly 
identifying learning outcomes in coursework and during advising to develop student 
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metacognitive awareness about the criteria for competence and to foster life-long learning; 
Intention - supports learning through articulating learning activities that offer opportunities to 
develop specific ELO competencies in syllabi and course aims; Utility - integration of specific 
ELO concepts and competencies within the course, among other courses, and in the real world to 
provide a sense that these competencies and concepts are meaningful; Reflection - providing 
opportunities to intentionally reflect upon their learning experiences along with meaningful 
learning experiences and specific evaluation criteria helps students to build awareness and learn 
more about their specific competencies. This model is useful for integrating ELOs into “courses, 
assignments, and co-curricular activities” (Cydis, et ah, p. 47). The current project illustrates how 
university level ELOs can be linked to program level outcomes (PLOs), and course outcomes 
(SLOs). Using a specific learning opportunity in the fonn of a multifaceted student learning 
experience operationalizes the concepts of ELO integration identified by Cydis and colleagues 
(2015). 

ELOs can “provide direction to faculty in designing courses, but to understand whether 
competency frameworks are affecting student learning outcomes, there must be some way to assess 
whether students can demonstrate the competencies” (Klein-Collins, 2012, p. 30). LEAP 
developed rubrics for assessment and several competency based college and university programs 
are approaching competency assessment through the use of rubrics (Klein-Collins, 2012). “A 
rubric is typically an evaluation tool or set of guidelines used to promote the consistent application 
of learning expectations, learning objectives, or learning standards in the classroom, or to measure 
their attainment against a consistent set of criteria” (Hidden Curriculum, 2014, para 1). Rubrics 
provide transparency and consistency in evaluation, clearly define expectations for student 
achievement before learning experiences occur, and allow for more specific, detailed feedback to 
students regarding their level of achievement. Rubrics also provide an opportunity for students to 
reflect upon and evaluate their own learning (Klein-Collins, 2012; Hidden Curriculum, 2014). 
Rubrics typically include the educational purpose of the assignment and links to appropriate 
concepts or learning objectives. They also include specific criteria that will be evaluated to 
demonstrate competence or achievement. Perhaps the most important component of a rubric is 
explication of specific quality standards that the instructor or evaluator will use in scoring, grading 
or evaluating the student’s performance. “For example, if the evaluation scale is scored “on a scale 
of 1 to 4, the rubric may detail what students need to do or demonstrate to earn a 1, 2, 3, or 4. Other 
rubrics will use descriptive language —does not meet, partially meets, meets, or exceeds the 
standard, for example—instead of a numerical score” (Hidden Curriculum, 2014, para 2). 

Linking Levels of Learning Outcomes and Competencies 

The university level ELO defines Ethical Reasoning as: “the recognition of an ethical dimension 
in a situation, that leads to critical examination of the potential consequences of your (in)actions 
based on consideration of alternate viewpoints and multiple principles and their implications”. 
Students are expected to gain knowledge in personal and social responsibility, including Ethical 
Reasoning throughout their college education, at successively higher levels. 

The Core Competencies of Interprofessional Collaborative Practice (Interprofessional 
Education Collaborative Expert Panel [IPCEP], 2011) serve as the theoretical framework of the 
BSHS program curriculum. One of these competencies is Values and Ethics. The BSHS Program 
Mission is to provide a quality, contemporary, pre-health professional education for students who 
desire careers in the healthcare system. The program curriculum consists of eight core course plus 
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several cognates and electives depending on the individual student’s educational goals and focus. 
There are 3 concentrations that prepare students for specific graduate programs such as physical 
therapy, plus a general concentration that prepares students for direct entry into the healthcare 
system or to pursue a variety of graduate programs. Current enrollment in the BSHS program is 
1111 undergraduate students. The BSHS program is the largest program in the university. One of 
the program outcomes (based on the IPCEP core competency of Values and Ethics) was that 
students critically discuss the interface of value/ethics on health outcomes. 

In the university level ELO of Ethical Reasoning, one criterion was specific to ethical reasoning 
in dealing with moral dilemmas. This ELO linked to specific program and course outcomes as 
follows (see figure 1): 

ELO: Demonstrate ethical reasoning when faced with moral dilemmas in a variety of real-world 
contexts 

PLO: Critically discuss the interface of value/ethics on health outcomes 

SLO: Articulate the impact of personal values and professional ethics in healthcare decision¬ 
making. 



Figure 1: Linking Essential Learning Outcomes 


Course Assignment 

In the second of two introductory BSHS program courses, one of the course student learning 
objectives (SLOs ) is to articulate the impact of personal values and professional ethics in 
healthcare decision-making. A learning activity designed to help meet this objective involved a 
multifaceted case study assignment. Students were given a case study that required a 
multidisciplinary healthcare team to make an ethical decision. Students formed groups of no more 
than four members. A variety of cases were provided from which each group chose one. Students 
in each group assumed the role of a specific member of the healthcare team, such as a physician, 
nurse, occupational therapist, or family member. They had to look up the codes of ethics of the 
professional they represented and any other pertinent legal or ethical standards. In class, several 
ethical decision making models were presented, and student groups were required to use one to 
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work through the ethical issues and come to a decision. Using the case study, they explored the 
relationship of values and ethics to making a healthcare decision. Students had to identify their 
personal values as well as those of the profession they represented and explore any conflict 
between the two. They also had to identify and resolve any conflicts within the group in order to 
come up with a resolution that satisfied all members. Each group created a presentation that was 
given to the class detailing the ethical dilemma and considerations, the decision they made, why 
they thought it was the best decision, and the process by which they arrived at the resolution. Each 
student wrote an individual scholarly paper with a reflective component based on this experience. 

Assessment Procedure 

Review of the ethical case study assignment criteria revealed that they linked quite clearly with 
the levels of competency in the Stockton ELO. After linking university ELOs, program PLOs, and 
course SLO’s, we chose to analyze the student case based ethical reasoning papers to evaluate the 
competency level at which students were operating by the end of the course. Student papers were 
received from all but one instructor and represented 3 of 5 sections of the course. Approx. 160 
students enrolled in HLTH 1102 during the time of this but 70 were inadvertently discarded by an 
instructor of 2 sections of the course. A total of 94 student papers were included in the analysis. 
This represented 59% of the total semester enrollment in the course, and 3 of 5 sections taught. 
University IRB approval was obtained prior to analysis. 

Papers were analyzed by a faculty member who did not know the students or teach the 
course. Papers were analyzed after the semester was over and did not affect or reflect student 
grades in the course or the assignment. The analysis was used to help determine how well students 
met the ELOs, PLOs, and SLOs related to Ethical Reasoning and the IPCEP Core Competency of 
Values and Ethics. A scoring rubric was developed based on the assignment guidelines which 
mirrored the college level ELO. See Figure 2 for the university competency levels of the Ethical 
Reasoning ELO. 

Five items in “Introduction” section guidelines of the student paper were clearly indicative of 
the "Aware" level of competency according to criteria in Stockton's Ethical decision making ELO. 
Each of these items were scored as either 0-absent or 1-present. 

1. Provides a brief summary of the facts of the case, including identifying the health problem(s), 
case needs, and key players. 

2. Clearly identify the ethical dilemma(s) in the case 

3. Present your personal moral positions in relation to the ethical dilemma(s). 

4. Present the ethical position of the profession you are representing in the IP team. Make sure to 
research and cite the Code of Ethics or your assumed professional role. You can also cite 
articles discussing your assumed profession’s position in this issue. 

5. Describe divergent moral (e.g., from patient, family) and/or ethical (i.e., professionals) 
positions in the IP group. If applicable, discuss the role and moral position in the group of 
patient, family, and friends. 
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Figure2: Stockton University Essential Learning Outcomes: Ethical Reasoning 


There were 3 items in the Ethical Decision Making section of the paper guideline that li nk ed 
to the ELO. These were analogous to the “Competent” level in the ELO. These were rated from 0 
to 2 with 0 indicating not met, 1 indicating partially met, and 2 indicating completely met. 

1. Describe completely at least two (2) recommendations discussed in the group, including their 
pros and cons, and anticipated outcomes. 

2. Describe how you & your team came to a decision between options considered in #2. 

3. Describe the agreed upon course of action and why it was the best option (support the reasons 
you give why it was the best option) 

Three items in the reflection section of the paper were linked to the “Skilled” level of the 
university ELO and were scored from 0 to 3, with 0 indicating not met, 1 indicating minimally 
met, 2 indicating partially met and 3 indicating completely met. 

1. Reflect and discuss agreements and/or disagreements between your personal moral position 
and the ethics of the profession you were representing (intrapersonal conflict). 

2. Reflect and discuss agreements and/or disagreements between you (personal morals and 
professional ethical position) and those of other group members (interpersonal conflict). 

3. Assess how well you feel you understood you own moral position in relation to the ethical 
dilemma. Assess how well you feel you understood the ethical position of the profession you 
were representing. Which one you understood better? How your understanding of each affected 
your performance in the group. 

A final item from the paper was linked to the Knowledge Transfer criterion of the ELO, 
representing the highest level of learning. It was scored from 0 to 3 as in the previous section. 

1. Summarize what this learning experience was like for you personally and in your future 
professional role. For example, how this experience may impact you if you experience a similar 
situation in your personal life and in your future professional role. 

Separate subscale totals were calculated for each of the 3 subscales representing Aware, 
Competent, and Skilled level criteria, plus one for Knowledge Transfer. 
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Assessment Results 

Using the narrative scoring rubric scores, means for each of the subscales were calculated. Relative 
scores were computed as percentage of the maximum possible score achieved. The mean score for 
the Aware level subscale was 4.67 (93.40%) out of a possible 5 points. The Competent subscale 
mean was 5.48 (91.33%) of a possible 6 points, and the Skilled subscale mean was 7.79 (86.55%) 
with 9 points as the maximum achievable score. The overall score was computed by adding the 
subscale scores for a maximum of 20 points. The overall mean score was 17.94 (89.70%). Results 
are displayed in Table 1. 

Table 1 

Narrative Analysis Rubric Scores 


h-94 

Subscale 

Scale 

Points 

Sample 

Mean 

Relative 

Score 

Sample 

Range 

Level 1: Aware 

0-5 

4.67 

93.40% 

3-5 

Level 2: Competent 

0-6 

5.48 

91.33% 

2-6 

Level 3: Skilled 

0-9 

7.79 

86.55% 

0-9 

Level 4: Knowledge Transfer 

0-3 

1.82 

60.66% 

0-3 

Overall Score 

0-20 

17.94 

89.70% 

8-20 


Discussion 

Results indicated that the majority of students achieved desired course and program outcomes 
related to Ethical Decision Making. The course level objective related to the core competency of 
Values and Ethics was fully met. Most students achieved the skilled level of the University ELO 
mid-way through program at the end of the second Introduction to Health Science course. Role 
playing and reflection were successful strategies in helping to achieve desires learning outcomes. 
Using a rubric to score student’s competency when analyzing papers on ethical decision making 
provided an accurate and authentic evaluation of student learning that can be used in determining 
how well university level, program level, and course level learning outcomes were achieved. 
Linking program and course level learning outcomes to university level ELOs was essential in the 
process. 

Results were as expected given students’ level of progression in the program and the university. 
The university ELO that students “Possess the ability to transfer the above skill to multiple areas 
such as personal, professional, and societal” was only partially met. This represents the highest 
level of learning. It is not possible to accurately measure this ELO in pre-professional students 
who do not have supervised clinical experiences 

Given students’ level in program, program goals related to values and ethics adequately 
achieved. These students have one more core program course in which Values and Ethics are 
included as an objective. Evidence from this assessment project suggests that students should fully 
achieve program outcomes in Values and Ethics by completion of the final core course. 

Our results suggest that most BSHS students should fully meet criteria for “Skilled” level in Ethical 
Decision Making according to college level ELO criteria by graduation. 

Linking the university, program, and course level ELOs illustrates the concepts of Intention 
and Facilitation in Cydis and colleagues’ (2015) model of ELO integration. The concepts of 


Journal of the Scholarship of Teaching and Learning, Vol. 17, No. 1, February 2017. 
josotl.indiana.edu 


21 



Reed, Garcia, Slusser, Konowitz, and Yep 


Awareness, Reflection, and Utility were demonstrated through the requirements of the assignment 
that students identify and reflect upon their own values and ethics in relation to those of others, 
reflect upon the process by which a decision was reached, and apply learning by role playing in a 
case study example similar to what may be encountered in the real world. The assignment itself 
facilitated mastery. The concept of Connection is illustrated by linking ELOs. PLOs, and SLOs, 
and in the intentional weaving of the competency of Values and Ethics throughout the BSHS 
curriculum as a course and program outcome. 

Care must be taken when appraising the results reported here since narrative analysis of student 
papers using a rubric is not a precise quantification. The rubric used in scoring is not nor was it 
intended to be a research instrument with precise psychometric qualities. It does provide an 
authentic assessment of student achievement of learning outcomes that can be used in course and 
program assessment related to Values/ Ethics. It was an appropriate choice for this project since it 
was used to measure the attainment of learning objectives against a consistent set of criteria 
(Hidden Curriculum, 2014). 

This strategy could be combined with a standardized objective measure of Values/ Ethics in a 
mixed methods design. This strategy has potential for use in assessing a variety of Student 
Learning Outcomes if closely linked with course, program, & college outcomes. 

References 

American Association of Colleges and Universities (AAC&U). (2015). The LEAP challenge: 
Education for a world of unscripted problems. Retrieved from: 
https ://www.aacu. org/leap/challenge 

American Association of Colleges and Universities (AAC&U). (2007). College learning for the 
new global century. Washington, DC: Author. 

Klein-Collins, R. (2012). Competency Based Programs in the U.S.: Post-secondary Credentials 
for Measurable Student Learning and Performance. Council for Adult and Experiential Learning: 
Chicago, IL. 

Cydis, S., Galantino, M., Hood, C., Padden, M., & Richard, M. (2015). Integrating and assessing 
essential learning outcomes: Fostering faculty development and student engagement. Journal of 
the Scholarship of Teaching and Learning 15(3), 35-52. 

Spady, W. (1994). Outcome based education: Critical issues and answers. Arlington, VA: 
American Association of School Administrators. 

Hakel, M. (2010). Learning goes beyond knowing. The Psychologist-Manager Journal, 13. 234- 
236. doi: 10 1080/108887136.2010.522475 

Hidden curriculum (2014, August 26). In S. Abbott (Ed.), The glossary of education reform. 
Retrieved from http://edglossary.org/hidden-curriculum 


Journal of the Scholarship of Teaching and Learning, Vol. 17, No. 1, February 2017. 
josotl.indiana.edu 


22 




Reed, Garcia, Slusser, Konowitz, and Yep 


Interprofessional Education Collaborative Expert Panel [IPCEP], (2011). Core competencies for 
interprofessional collaborative practice: Report of an expert panel. Washington, D.C.: 
Interprofessional Education Collaborative 

Malan, S. (2000). The ‘new paradigm’ of outcomes based education in perspective. Journal of 
Family Ecology and Consumer Sciences, 28, 22-28. 

Stockton University (n.d.). Retrieved from: http//www. Stockton 

Slade, S. & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas. American 
Behavioral Science, 57(10), 1510-1529. 

Sullivan, D. (2015). The VALUE breakthrough: Getting the assessment of student learning in 
college right. Washington, DC: AAC & U. 

Teixera-Poit, S., Cameron, A., & Schulman, M. (2011) Experiential learning and research ethics: 
Enhancing knowledge through action. Teaching Sociology 59(3). 244-258. doi: 

10:1177/0092055X11407346 


Journal of the Scholarship of Teaching and Learning, Vol. 17, No. 1, February 2017. 
josotl.indiana.edu 


23 



